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APPROXIMATION  OF  DELAY  SYSTEMS  WITH  APPLICATIONS 
TO  CONTROL  AND  IDENTIFICATION 


H.  T.  Banks 

\1 

Abstract:  We  discuss  approximation  ideas  for  functional  differential 
equations  and  how  these  ideas  can  be  employed  in  optimal  control  and 
parameter  estimation  problems.  Two  specific  schemes  are  described, 
one  based  on  integral  averages  of  the  function  being  approximated, 
the  other  based  on  best  L^jspline  approximations.  An  example 
illustrating  numerical  behavior  of  these  schemes  applied  to  an  optimal 
control  problem  is  presented. 
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INTRODUCTION 

In  this  presentation,  we  shall  discuss  approximation  ideas  that 
have  proved  useful  in  developing  methods  for  solving  optimal  control 
and  parameter  identification  problems  involving  delay  systems.  We 
shall  focus  on  1 inear  delay  systems  in  our  discussions  although,  as  we 
shall  later  indicate,  many  of  the  ideas  and  results  are  valid  for 
problems  with  nonlinear  systems. 

The  approximation  results  we  describe  below  are  based  on  an  ab- 
. stract  formulation  due  to  Trotter  and  Kato  dealing  with  approximations 
to  semigroups  of  linear  operators.  In  order  to  make  use  of  these 
approximation  theorems,  it  is  necessary  to  reformulate  our  delay  system 
as  an  abstract  system  in  an  appropriate  Hilbert  space.  To  this  end, 
consider  the  delay  system 


x(t)  = L(xt)  + f(t)  0 < t < t1, 

x(0)  = n,  xQ  = 4>, 


where,  for  <P  continuous  on  [-r,0], 


v 

i = 0 1 


+ 


[ o 

A(0)^(O)dO 


CD 


(2)  . 


with  A^,  A(<3)  given  n * n matrices  and  0 = < < Tv  < r. 

For  0 < t < t^,  the  function  9 -*■  xt(6),  -r  < 6 < 0,  is  given  by 
xt(9)  = x(t+0)  whenever  x is  defined  on  [-r,t^]. 

With  an  appropriate  interpretation  of  the  operator  L,  one  can 
define  solutions  x (i.e.  solutions -exis-t)  to  (1)  corresponding  to 
initial  data  (n,*)  in  2 = Rn  * L2( [-r, 0] ,Rn)  s Rn  * Lj(-r,0)  and 
perturbations  f in  L^CO.t^)  as  functions  which  satisfy  the  initial 
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conditions  in  (1),  are  absolutely  zontinuous  on  (.(J,t.,)  and  satisfy 

i 

the  differential  equation  in  (1)  almost  everywhere  on  (0,t^).  One 
can  thus  define  a homogeneous  solution  semigroup  of  operators 
S(t)  : Z -*•  Z,  t > 0,  by 


s(t)Cn,<j>)  = (x(t;n,<lO 


where  x is  the  solution  of  (1)  with  f = 0.  It  is  not  difficult  to 
argue  that  this  defines  a strongly  continuous  semigroup  (Cq- semigroup) 
{S(t)>t>Q  of  linear  operators  with  infinitesimal  generator  (i.e. 

S(t)  ~ on  = { (<f>  (0)  ,<f>)  |<(>  is  absolutely  continuous 

with  in  L2(-r,0)}  given  by 

J3^(*  CO)  ,«(►)  = CL(*),J).  (3) 


This  semigroup  can  be  used  to  give  an  abstract  "variation  of  para- 
meters" representation  for  solutions  of  (1)  in  Z.  That  is,  suppose 
one  defines  for  (n,<lO  e Z and  f e L^fO,^)  the  function 
- t‘  -*■  z ( t ; n , , f ) by 

z C t ; n , •?> , f ) = SCt)  Cn,4>)  + S ft- a)  (f  Co) , 0)  do.  (4) 

0 

Then  one  can  argue  (see  [2])  an  equivalence  between  (1)  and  (4).  Mor 
precisely,  one  has  that 


z(t;  n,(f>,f)  = (x(t;n,<t>,f) , xt(n,<t>,f)) 

for  t > 0 where  z is  given  by  (4)  and  t x(t;n><?>f)  is  the 
solution  of  (1)  . 

Tor  n = <»>  C 0 ) and  4> , f sufficiently  smooth,  it  turns  out  that  (4) 
is  equivalent  to  an  ordinary  differential  equation  (ODE)  in  the 
Hilbert  space  Z which  may  be  written 


z(t)  = J*z(t)  + (f (t)  ,0) 
z(0)  = (<K0)  , <}>)  . 


(5) 


We  seek  to  approximate  (4)  (or  (5))  by  considering  approximations 

N 

in  finite  dimensional  subspaces  Z (where  we  will  obtain  finite 
dimensional  ODE's).  The  fundamental  approximation  result  from  semi- 
group theory  that  we  employ  can  be  stated  roughly  as  "if  - 3^ 


■ 


u/  £ f/t 

in  a proper  sense,  th.n  c ->  in  some  sense'.  One  version  of 

this  result  due  to  Trottcr-Kato  11.3]  can  be  stated  in  precise  terms 
as  follows. 

Theorem.  Suppose  are  generators  of  Cn -semigroups  {SN(t.)}, 


(S(t)}  on  Z satisfying: 

(i)  there  exist  M and  0 > 0 such  that 


|SN(t)  | < Me  et,  |S(t)  | < Me  St , 


(ii)  there  exists  a dense  subset  ^ of  Z such  that  J2?  z **■  £&'z 
as  N oo  for  each  z e 3>t 

(iii)  there  exist  a complex  number  with  real  part  larger  than 

0 such  that  is  dense  in  Z. 

N 

Then  S (t)z  ■+•  S(t)z  as  N ■+  °°  for  every  z e Z,  uniformly  in  t on 
compact  intervals. 

We  wish  to  emphasize  that  use  of  this  semigroup  formulation  is 

mainly  for  convenience  in  proving  convergence  of  particular  schemes. 

Just  as  in  the  case  of  differencing  schemes  for  solution  of  partial 

differential  equations,  the  semigroup  formulation  is  not  an  essential 

aspect  in  the  development  of  numerical  methods.  Rather,  the 

N 

pai ticular  choice  of  the  subspaces  Z and  the  approximating  operators 
N 

is  the  question  of  utmost  importance. 

In  both  the  control  and  identification  applications  discussed  here, 

the  basic  system  to  be  considered  is  (1)  or,  equivalently  (4).  Once 

N yN 

we  have  chosen  Z and  satisfying  the  Trotter-Kato  hypotheses, 

N N 

we  shall  also  employ  (orthogonal)  projections  P of  Z onto  Z to 
then  approximate  equation  (4)  by 

zN'(t)  = SN(t)PN(n,<f>)  + ftSN(t-cr)PN(f  (o)  ,0)dcr.  (6) 

0 

N \T 

Since  in  the  cases  of  interest  to  us  _Q/  will  be  bounded  and  S‘  (t) 
J3fNt 

= e , we  may  equivalently  write  this  as 

v 

zN(t)  = ^NzNCt)  + PN(f(t),0)  C7) 

zN(0)  = PN(n,<t>). 

N 

The  convergence  of  S to  S will  be  sufficient  to  guarantee  a 
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desired  convergence  of  solutions  < f'  (7)  in  both  the  optimal  control 
and  parameter  identification  problems  detailed  below. 

OPTIMAL  CONTROL  PROBLEMS 

The  basic  idea  employed  in  control  problems  is  a classical  one 
which  is  the  basis  for  all  Ritz  type  methods  in  approximation  theory. 
Suppose  one  has  an  optimal  control  problem  (^)  in  a Hilbert  space  Z. 

By  considering  a sequence  of  approximating  problems  ( on  finite 

N N 

dimensional  subspaces  Z , one  seeks  to  obtain  a sequence  of  more 

easily  solved  problems  whose  solutions  will  approximate  (i.e., 

approach  in  the  limit  as  N -►  »)  solutions  of  the  original  problem 

( &) . To  be  more  specific,  let  f(t)  = Bu(t)  1 n (1)  where  B is 

an  n * m matrix  and  u is  to  be  chosen  from  some  admissible  class 

C L^fOjt^)  of  control  functions.  The  problem  ( 5?)  might  typically 

consist  of  choosing  u e ^ so  as  to  minimize  (we  do  not  distinguish 

between  a vector  and  its  transpose  here  and  below) 

■ A 

4>  Cu)  = x(t,)Qx(t,)  + (x(t)Wx(t)  + u(t)Ru(t)  }dt,  (S) 

J0 

where  x is  the  solution  of  (1)  corresponding  to  u and  the  matricc- 
Q,W  and  R are  symmetric  with  Q and  W positive  semi-definite,  R 
positive  definite. 

N n 

Writing  solutions  z of  (7)  in  terms  of  components  in  R and 

L ^ = (xN,yN),  and  defining 

Au)  = xN(t1)QxN(t1)  + ftl{xN(t)WxN(t)  + u (t) Ru ( t) } dt , (9) 

1 1 0 

» 

N 

we  take  as  our  approximating  problem  ( j-^)  that  of  minimizing  $ 
over  subject  to  (7). 

If  we  denote  by  u,uN  solutions  to  problems  {<?)  and  ( ^) 

respectively,  one  can  often  use  the  Trotter-Kato  results  to  guarantee 

— N — 

convergence  of  the  u to  u in  a desired  sense.  For  example,  in 

the  case  of  given  by  (8),  (9)  and  a convex  closed  set,  one 

— — N 

can _argue  that  the  optimal  controls  .u,u  „ exist  and  are  unique, 

— N — N — M — 

u u in  L2  as.  N -*■  «>,  and  furthermore  $ (u‘  ) •>■  $(u).  Similar 

results  can  be  obtained  for  more  general  payoff  functions  in  place  of 

(8)  and  (9).  For  details  along  with  convergence  arguments  see  [2], 

[3J  . 


iUi.,U  I i lL’A'i  ION  PROBLEMS 
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In  many  practical  situations,  it  is  important  to  be  able  to  fit 
a model  described  by  equations  such  as  (1)  to  data  obtained  through 
empirical  efforts.  In  this  case  one  has  certain  parameters,  say 

k 

q e R , on  which  the  system  depends  and  which  must  be  "identif icd"  or 
"estimated".  for  example  suppose  the  operator  L in  (2)  depends  on 
parameters  q (typically  these  may  be  some  of  the  matrix  coefficient 
terms  in  (2)  and/or  even  a delay  so  that  L might  have  the  form 
L(q;i|0  = Ag(a)^(0)  + A^  (a) ’•)>(- t)  , q = C 01 » 3 • One  then  must  identify  q 
in  the  system 

x(t)  = L(q;x  ) + f(t) 

z (JO) 

c(t)  = Cx (t) . 

Here  c(t)  represents  the  "observables"  for  the  system.  One  pe:turbs 
the  system  (via  f)  and  collects  data  £^,...,£^  at  times  t^,...t^ 
(the  are  observations  for  c(t^)).  The  problem  is  to  choose  q 

from  some  admissible  parameter  set  Q so  that  it  is  a maximum 
likelihood  estimator  (MLE)  or  perhaps  so  that  it  minimizes 

M 2 

J(q)  = l 1 c (t . ; q)  - S-r.  HD 

1=1  1 1 

Just  as  in  the  control  problems,  one  reformulates  this  identifica 
lion  problem  as  one  in  the  abstract  space  Z involving  a system  such 
ajj  (4)  (or  (5)),  and  approximates  by  a sequence  of  identification 
problems.  That  is,  one  has  approximating  systems  (again,  z^'  = (x^ , y^V) 

At)  = J*Aq)  At)  + PN(f(t),0) 

N N (12) 

cN(t)  = CxN(t) 

with  observations  {?.}  for  (c^(t.;q)}.  One  tlien  chooses 

\j  ^ ^ 

q‘  £ £ C R so  that  it  is  a MLE  for  (12)  or  so  that  it  minimizes 

Aq)  = l |cN(ti;q)  - C.|2,  (13) 

i = 1 

depending  op  how  one  has  decided  to  seek  a fit  to  data  for  the 
original  system. 

Under  quite  reasonable  assumptions,  one  can  use  the  Trotter-Kato 
results  to  guarantee  existence  of  a vector  q (loosely  speaking 
lim  <A  so  that  i^(q)  is  the  limit  of  ja^Aq^)  in  an  appropriate 


sense  and  so  that  q is  a MLE  (or  a minimizer  for  (11)  if  (13)  is  used 
in  the  approximating  problems)  for  (10)  with  data 

There  remains  the  basic  question  of  how  to  choose  ZN  and  jq/‘' 
so  that  the  needed  Trotter-Kato  hypotheses  are  satisfied  and  most 
importantly,  so  that  efficient  numerical  schemes  are  generated.  l\'e 
shall  discuss  and  compare  two  particular  choices  here;  the  first  we- 
refer  to  as  "averaging"  (AVE)  approximations  while  the  second  involves 
spline  type  (SPL)  approximations. 

AVERAGING  APPROXIMATIONS 

For  a given  positive  integer  N,  one  partitions  the  interval 
(-r,0)  into  N equal  subinterval s with  a partition  {tj},  tj  = -jr/N, 
and  defines  the  characteristic  functions 

X i = X ^ ^ » j = 2 , . . . , N , X,  = X jq  j i • 

N 

One  then  takes  Z to  be  the  n(N+l)  dimensional  subspacc  of  l 
defined  by 

ZN  . {(».*)  | n e Rn,  '*  - I V W,  vN  c ,)n) . 

j=l  3 3 J 

N N 

The  projections  P : Z ->  Z are  given  by 

N 


where  = (N/r) 


„N,  ...  , V N, 

P (n,40  = (n,  l <j>  • x ; ) 

1 3 3 


~1(Ks)ds,  the  integral  averages  of  <p  over  the 


tN 
J ■ 


partition  subintervals.  Finally,  one  defines  the  approximating 

MM 

operators  stf  : Z -*■  Z by 


N 


^ ,rN,  *>  c N,.N  N'  N> 

sz/  (n,<p)  = (L  (n,4>),  I 

j = 1 r 3 1 33 


N 

where  4>0  . = n and 


N 


v N 


l"CM)  s A„n  ♦ £ l uW(-ti)  . J A>" 


N 


iN.N 


i-1  j-1 


i jnr  i' 


j-i 


j j 


with  A^  = 
J 


tN 

(V  i 

tN 
t3 


A(6)d9.  With  these  definitions  of  one  can 


. 


then  argue  that  the  desired  Trotte ■.  - Kato  hypotheses  are  satisfied  ar.d 
moreover  that  the  convergence  beha/ior  necessary  for  use  in  both 
optimal  control  and  identification  problems  is  obtained  (see  [3], [8]) 


SPLINE  APPROXIMATIONS 

For  order  spline  approximations,  one  defines  the  subspaco: 

ZN  = f N)  | is  a spline  of  order  K on  [-r,0]  with  knots 

t^  = -jr/N,  j 3 0,1,,.., N).  Letting  P^  be  the  orthogonal  pro- 

* . M 

jection  of  Z onto  the  closed  subspace  Z , one  can  then  define 

J£/N  = PNjz?PN 

where  s/  is  given  in  (3). 

For  example,  in  the  case  of  first  order  ("linear  elements") 
splines,  the  subspace  Z^  is  of  dimension  n(N+l).  Jf  one  defines 


the  usual 

"roof" 

functions 

; for 

j = 

rH 

O 

N 

and  -r  < 8 < 0 by 

N 

r 

t*-*J*X> 

tN 

j + 1 

< 8 < t^ 
- 1 

e^(9) 

= < 

N 

r 

Ctj.l-f) 

/ 

tN  < 

J ' 

8 < tN 
- j -1 

1 

L 

0 

elsewhere , 

. N 

where  t . 

J 

= -j 

r/N 

for  all 

j > 

then 

ZN  can 

be 

written 

— 

Z 

N 

kAo), 

A I 

*N  - 

r N N 
l a.e.. 

N 
a . 

e Rn}. 

j=0  J J 

J 

Of  course, 

pn( 

n,4>)  = <T  = 

(Ao),A 

where 

<-N 

<P 

is  the  solution  to 

problem  of 

min 

imi: 

zing  Z 

- (n, 

<0)1 

over  Z 

in 

ZN. 

The  conditions  for  the  Trotter-Kato  theorem  can  be  verified  .for 
'these  spline  based  approximations  and,  in  fact,  the  arguments,  using 
fundamental  estimates  from  spline  analysis,  are  independent  of  the 
order  of  splines  used  (see  [6]). 

For  both  the  AVE  and  SPL  approximations,  one  can  obtain  estimates 

on  the  convergence  rates  of  the  approximations.  The  AVE  method  is 

essentially  a first  order  method  (error  like  1/N)  while  one  can  argue 

t h v 

that  the  error  in  the  K order  SPL  scheme  behaves  like  1/N  . Ir. 

actual  practice  our  computations  have  revealed  that  the  SPL  methods 

usually  converge  more  rapidly  than  these  estimates  predict  (sec  the 

discussions  in  [6])  and  indeed  for  both  control  and  identification 


proorcms  it  appears  that  the  "fir;  order"  SPL  method  often  offers 
significant  advantages  over  the  /'a  method  at  little  cost  in 
additional  complexity  with  respect  to  implementation.  To  partially 
illustrate  this  feature  and  to  indicate  what  one  might  typically  ex- 
pect in  the  way  of  convergence  for  these  methods,  we  present  just  one 
of  a number  of  control  examples  to  which  we  have  applied  these  ideas 

EXAMPLE  (CONTROLLED  OSCILLATOR  WITH  DELAYED  DAMPING) 

The  problem  is  to  minimize 

2 

*(U)  = 5x^2) 2 + \ f u(t)2dt 
over  u e - L ^ ( 0 » 2 ) subject  to 

. I 

X. (t)  = X-(t) 

1 (14) 

x2(t)  = -X1(t)  - x 2 ( t - 1 ) + u(t),  0 < t < 2, 


x1(0)  = 10 

x2(0)  = 0,  -1  < 9 < 0. 

The  system  (14)  is,  of  course,  the  vector  formulation  for  the  system 
dynamics  y(t)  + y(t-l)  + y(t)  = u(t).  One  can  use  necessary  and 
-Sufficient  conditions  (a  maximum  principle  for  delay  system  problems j 
to  solve  analytically  this  simple  example.  Upon  doing  so  one  finds 
(see  [3],  [5])  the  optimal  control 


u(t)  = 


6 sin(2-t)  + (6/2) (1-t) sin(t-l) 
6 sin(2-t) 


0 < t < 1 


1 < t < 2 , 


where  6 is  the  solution  of  a certain  algebraic  equation  with 
approximate  value  6 ~ 2.5599  . The  optimal  value  of  the  payoff 
given  by 


4>  = *(u)  3.399119. 

We  used  a conjugate-gradient  scheme  to  compute  the  solution  of  each 
of  the  approximate  control  problems  ( ^N)  described  above  for  both  the 
AVE  and  SPL  schemes  for  several  values  of  N.  Denoting  the  optimal 
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values  ^N(^ve)  and  $N(Ugp]  ) of  the  payoffs  by  and  ?gpi  re- 
spectively, we  give  a representative  sample  of  our  numerical  finding- 
in  tabular  form  . 


N 

-rN 

*AVE 

1 ^AVE'4*! 

-rN 

*SPL 

,-rN  * 

1 *SPL‘* 

1 

4 

2.15154 

1.2475 

3.53549 

.1363 

8 

2.67110 

.7280 

3.43454 

.0354 

16 

3.00356 

. 3955 

3.40857 

.0094 

i 

32 

3.19298 

. 2061 

3.40193 

. 0028 

\ 

. i 


We  note  that  the  convergence  of  $ 


-rN 


that  of 


AVI: 


to  4>  is  like  1/N  while 


-N  -N 
USPL,UAVE 


is  1/N  . A graph  comparing  several  of  the  optimal 

with  the  solution  u of  the  original  problem  is 


controls 

given  in  Figure  1.  One  also  has  for  this  example  that 


u 


rN  , 

SPL'L. 


like  1/NZ  while 


I“ave'l. 


I u j T is  like  1/N. 
“2 


( _ - We  complete  our  presentation  with  brief  mention  of  related  results 

1 that  have  been  obtained  through  efforts  by  our  group  at  Brown 

University  and  our  associates  and  colleagues. 

The  basic  abstract  framework  employing  the  Trotter-Kato  theorem 
i to  ensure  convergence  in  optimization  problems  for  delay  systems  was 

given  in  [2].  A detailed  investigation  of  the  AVE  approximation  and 
its  use  in  linear  system  control  problems  along  with  several  solved 

I -*■- 

examples  can  be  found  in  [3]  . Additional  examples  of  solved  control 
j problems  along  with  use  of  the  AVE  scheme  on  these  examples  may  be 

| found  in  [5].  An  extension  of  the  theory  of  [2],  [3]  to  treat  control 

problems  with  nonlinear  systems  along  with  related  nummerical  findings 
j are  given  in  [1] . In  efforts  to  find  alternatives  to  the  AVE  scheme, 

'!  Burns  and  Cliff  discuss  in  [7]  a scheme  involving  piecewise  linear 

approximations  combined  with  AVE  type  ideas.  Spline  approximations  in 
the  context  of  a general  framework  suitable  for  use  in  control  and 
identification  problems  are  developed  in  [6] . More  recently,  Kappel 
• and  Schappacher  [10]  have  used  interpolating  splines  for  an  approxima- 

tion method  that  is  applicable  to  nonlinear  retarded  equations  while 
^ Kunisch  [12]  has  developed  the  AVE  scheme  for  nonlinear  neutral  func- 

tional differential  equations.  Kunisch's  development  is  in  the  spirit 
, 1 of  the  earlier  results  of  Kappel  and  Schappacher  [9]  who  formulated  a 

"local  semigroup"  approach  to  handle  AVE  approximations  for  locally 

i 

1 Lipschitz  nonlinear  retarded  equations.  In  [8],  Burns  and  Cliff 


* 


- 
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discuss  the  use  of  the  AVI-1  scheme  in  identification  problems  while 
results  for  SPL  based  identification  methods  arc  presented  in  (4] . 

All  of  the  above  schemes,  whether  for  AVE  or  SPL,  lead  to  the 
approximation  of  a differential  equation  (e.g.  (5))  in  the  Hilbert 
space  Z by  a finite-dimensional  ODE.  To  actually  use  these  method; 
in  computations,  one  needs  a second  approximation  (e.g.  a standard 
Runga-Kutta  or  predictor-corrector  scheme)  to  solve  the  approximating 
ODE's.  A natural  question  that  arises  is  "why  not  go  directly  from 
the  original  (infinite  dimensional)  delay  system  to  a different 
equation?"  Reber,  in  [14],  [15],  investigates  this  question  in  the 
spirit  of  the  functional  analytic  methods  of  the  papers  cited  above 
(factor  space  idcas-see  [11] -are  employed  in  place  of  the  Trot  ter  - Ku to 
semigroup  results).  Briefly,  Reber  considers  quite  general  non  - 
autonomous  linear  delay  equations  and  shows  that  they  can  be  formulated 
as  an  abstract  system  in  Z 


Tz  = I(C,f) 


(IS, 


s 


j 

) 

i 


4 

I 


1 


where 


(Tz)  (t)  = 

z(t)  - 

j3f(o)z(o)da 

t0 

ft 

X(c,f)(t) 

= C + 1 

(f(a),0)da. 

J 

t0 

Here  S is  the  initial  data 

(n,4>) 

and  f is  the  perturbing  func 

tion  as  in  (1)  or  (5).  Equation  (15)  is  then  approximated  by 

rNzN  = £nPN^,£) 

where  T^  is  essentially  a first  order  difference  operator  and  the 
PN  corresponds  to  an  AVE  type  approximation  in  the  state.  This  re- 
sults in  a simultaneous  discretization  of  both  "state"  and  "time". 
Reber  discusses  the  use  (advantages  vs.  disadvantages)  of  these  ideas 
in  ^control  problems  (theoretically,  ‘the  analysis  is  slightly  more 
complicated  than  that  for  the  simple  "state"  discretization  - i.e.  ODE 
approximations  - ideas  discussed  above;  numerically,  implementation 
on  the  computer  is  quite  straightforward) . 
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[6] 

[-7] 

1 

18] 

[9] 

! no] 

i 

[11] 

[12] 

. [13] 
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